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Abstract 

In recent years, various state education departments have recommended the use of portfolios, an 
alternative assessment measure, as a means by which teacher education programs focus, guide 
and document teachers’ development of specific standards for performance. This paper describes 
the design and development of a student assessment portfolio for teachers’ professional 
development, as part of a course on the evaluation and assessment of students. The process used 
to establish categories and standards for assessment of multi-ethnic and racially diverse learners 
is outlined. Also presented are preliminary data on teachers’ perceptions of their knowledge of 
assessment, before and after, a course on evaluation and assessment of students, and teachers’ 
perceptions of how preparation of an assessment portfolio influences their assessment literacy. 
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Meeting the Challenges to Urban School Reform: 

Assessment Portfolios for Teachers’ Professional Development 
In the context of urban school reform, the prevailing view is that an interrelationship 
exists between all students' academic performance, including students who are educationally at- 
risk, and teachers’ academic standards, professional preparation and accoimtability. This 
perspective has led to several recent and significant educational initiatives affecting the teaching 
-learning experience of both students and teachers. For students, government agencies at 
national, state and local levels have developed curriculum standards to "raise existing standards" 
to foster higher student achievement and instituted new testing programs to monitor the quality 
of the teaching-learning experience. In such high stakes environments, teachers’ skills in 
measuring and monitoring students' achievement are increasingly challenged and become 
paramount and essential to the teaching-learning process. This greater emphasis on student 
testing and assessment represents a type of accountability measure where low test scores lead to 
potential negative consequences for teachers, students and school districts. Similarly, for 
teachers, state educational agencies have also developed other accountability measures such as 
teacher certification examinations to measure teachers' professional competencies. Teacher’s 
scores, high and low, on these examinations may serve as indicators of the quality of university- 
based teacher education programs. 

However, much controversy abovmds on the use of a single test score as a measure 
of either student achievement or teacher competency. Consequently, in recent years, educational 
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measurement specialists have continually explored alternative assessment measures. One 
alternative assessment approach that has received much recent attention, in the context of school 
reform, is portfolio assessment. Teachers have been encouraged to use portfolios as a 
performance-based strategy to monitor and assess progress of their students over time. Various 
state education departments have mandated the use of portfolios as a means by vv^hich teacher 
education programs focus, guide and document teachers' development of specific professional 
standards. In addition, these portfolios would be used as evidence of teachers' expertise to 
inform hiring, performance and other personnel decisions. The purpose of this paper is to: (a) 
discuss the design and development of a student assessment portfolio for teachers' professional 
development, as part of a course on the evaluation and assessment of children; (b) describe the 
process of establishing categories and standards for assessment of multi-ethnic and racially 
diverse learners; (c) outline the artifacts or forms of evidence of standards-based professional 
development in the area of assessment of multi-ethnic and racially diverse learners; and, (d) 
present preliminary data on teachers' perceptions of their knowledge of assessment, before and 
after a course on evaluation and assessment of children, and teachers' perceptions of how 
preparation of an assessment portfolio influences their assessment literacy. 

Design and Development of a Student Assessment Portfolio for Teachers' 

Professional Development 

Research has shown that, although a high proportion of teachers' professional practice is 
spent on assessment-related activities (Stiggins, 1991), most teachers report feeling ill-prepared 
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(Ward, 1980) and lack the competencies necessary to assess students' progress in the attainment 
of instructional objectives. This may compromise their effectiveness in the teaching-learning 
process (Plake & Impara, 1998). A growing body of research, specifically examining teacher 
assessment literacy, indicates that teachers receive little or no formal assessment training in 
college preparatory programs and find assessment strategies in measurement courses inadequate 
for use in the classroom (Schaffer, 1993). 

A recent collaborative effort of the American Federation of Teachers (AFT), the National 
Educational Association (NEA) and the National Council on Measurement in Education 
(NCME), developed and published seven "Standards for Teacher Competence in the Educational 
Assessment of Students” (AFT, NCME, NEA, 1990) A national survey was subsequently 
imdertaken to measure, using a multiple-choice test, practicing teachers' actual knowledge of the 
seven competency areas, as specified in the standards. The results of the survey indicated low 
levels of assessment competencies for teachers, including teachers exposed to measiarement 
coiusework and with the longest teniue (Plake & Impara, 1 998). Most importantly, this study 
strongly suggests that many teachers currently employed in various educational systems 
throughout the United Stated lack skills in the educational assessment of students. It also 
highlights the need for in-service training or other professional development strategies. Further, 
the school reform movement has increased the demand for teacher assessment literacy in 
monitoring and measuring student achievement, although teachers may not be adequately 
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prepared to assess students (Plake & Impara, 1998). Thus, as part of a course on evaluation and 
assessment of children, a student assessment portfolio assignment was designed to guide and 
document teachers' professional development in the area of student assessment literacy. 

In recent years, teaching portfolios have been used increasingly as an instructional tool in 
teacher education programs, to document teachers' professional accomplishments attained over 
time (Wolf, 1996). The essential components of teaching portfolios include artifacts (e.g. 
documents related to teachers' and students' achievement) and written reflections. The format 
and design of the student assessment portfolio assignment, in the present paper, were adapted 
from Wolf (1996). As shown in Table 1, the student assessment portfolio assignment consisted 
of the following: (a) decorative cover page, on a three-ring loose-leaf view binder, related to a 
selected topic, (b) table of contents, (c) introduction of teacher, (d) description of target 
population, in terms of age and/or grade level, to be assessed and selected curriculum umt or 
topic, (e) preparation of intended learning outcomes (i.e. instructional goals and specific 
objectives) based upon Bloom's taxonomy in the cognitive domain, (f) preparation of test items 
for each instructional objective, (g) preparation of test specifications, (h) preparation of a 
checklist or rating scale for a subset of test items to monitor attainment of the instructional 
objectives, (i) children's completed tests and performance assessments, (j) reflective summary of 
children's performance on the test and performance assessment constructed by the teacher, and 
the extent to which the objectives were attained, and (k) reflective summary of the teachers' self- 
assessment of his/her skills employed in developing the test and performance assessment. The 
teachers developed the student assessment portfolio over the duration of the course for 




7 



Assessment Portfolios and Teachers 7 



approximately a period of three months. All parts of the assignment were reviewed and approved 
by the instructor prior to completion. It is evident, therefore, that this is a time-consuming and 
labor-intensive process for both the student and the instructor. 

Process of Establishing Categories and Standards for Assessment of Multi-Ethnic 

and Racially Diverse Learners 

Two sets of standards, as shown in Table 2, were used to establish the categories and 
standards for assessment of multi-ethnic and racially diverse learners. The first set of standards 
was the seven "Standards for Teacher Competence in the Educational Assessment of Students" 
that was developed and published collaboratively by the American Federation of Teachers 
(AFT), National Educational Association (NEA) and the National Council on Measurement in 
Education (NCME) (1990). These standards are summarized as follows: (a) “Choosing 
assessment methods appropriate for instructional decisions” (Gallagher, 1 998, p.477) which 
includes use of measurement concepts of validity and error, knowledge of the relationship 
between assessment, instruction and making decisions about students, consideration of cultural, 
social, economic and language backgrounds of students, knowledge of resources for gathering 
information about and reviewing various assessment approaches and instruments, and various 
assessment options; (b) “Developing assessment methods appropriate for instructional decisions” 
(Gallagher, 1998, 478) which includes following appropriate measurement principles to develop 
and use assessment methods in teaching, selecting methods and techniques appropriate the 
teachers' intended learning outcomes and independently using student information to analyze the 
quality of assessment techniques; (c) “Administering, scoring, and interpreting the results of both 
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externally produced and teacher-produced assessment methods” (Gallagher, 1998, p. 479) 
which includes knowledge of standard scores and basic statistics, (d) “Using assessment results 
when making decisions about individual students, planning, teaching, developing cumculum, 
and school improvement” (Gallagher, 1998, p. 479); (d) “Developing valid pupil grading 
procedures which use pupil assessments” (Gallagher, 1998, p. 479); (e) “Communicating 
assessment results to students, parents and other lay audiences and other educators” (Gallagher, 
1998, p. 479); and (f) “Recognizing unethical, illegal and otherwise inappropriate assessment 
methods and uses of assessment information” (Gallagher, 1998, p. 480) ( AFT, NCME, NEA, 
1990). 

The second set of standards used to establish categories and standards for assessment of 
multi-ethnic and racially diverse learners was the American Psychological Association Standards 
for Educational and Psychological Tests (1974), Standards for Educational and Psychological 
Testing (1985) and Code of Fair Testing Practices in Education (1988). 

Artifacts/Forms of Evidence of Standard-Based Professional Development 
in the Area of Assessment of Multi-Ethnic and Racially Diverse Learners 
The content of the course on evaluation and assessment of students was also based upon 
the AFT, NCME, NEA (1990) standards for teacher competence in the educational assessment of 
students and APA (1974, 1985, 1988) standards for educational and psychological tests. The 
course was designed to explore eight major areas: (a) basic concepts and principles of 
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measurement and assessment, (b) preparation of instructional goals and objectives, (c) concepts 
of validity and reliability and other desired test characteristics, (d) classroom test construction, 
(e) measurement of complex achievement, (f) elementary statistics, (g) administration, use, and 
interpretation of standardized test and (h) alternative forms of assessment. The students' actual 
knowledge in these areas was assessed with a multiple-choice and essay midterm and final 
examination. The student assessment portfolio assignment added a standards-based and 
performance-based assessment as another dimension to the development of teacher assessment 
literacy. A performance assessment form was developed to monitor, guide and assess students' 
progress in completion of their student assessment portfolios. The artifacts or forms of evidence 
to be included in the portfolio, as shown in Table 3, were as follows: (a) the construction and 
administration of a test and performance assessment including the selection of a curriculum umt 
or topic, (b) preparation of learning outcomes (i.e. general goals and specific instructional 
objectives based upon Bloom's taxonomy), (c) preparation of test item, (d) demonstration of 
content validity, (e) preparation of a test blueprint, (f) development of a checklist to monitor the 
attainment of objectives, (g) preparation of a checklist or rubric for the performance assessment 
and (h) reflective summary of the children's performance on the assessment procedures, 
attainment of the objectives and the teachers' self-assessment of his/her skills in developing the 
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Preliminary Data on Teachers' Pre- and Post- Perceptions of Their Knowledge of 
Assessment and Preparation of a Student Assessment Portfolio 

As part of the course on evaluation and assessment of children, the instructor modeled the 
assessment and instruction process. On the first day of class, a course syllabus, consisting of the 
content to be covered in the course and the goals and objectives of the course, were distributed to 
and discussed with the class. After discussing the course goals and objectives, the class 
participants were administered the Evaluation and Assessment of Children Self-Assessment 
Questionnaire (EACSAQ), which was designed by the instructor. The EACSAQ consisted of six 
assessment content areas: 

1 . lam knowledgeable of the basic concepts and principles in measurement and 
assessment. 

2. lam knowledgeable of the role of measurement and assessment in the instructional 
Process. 

3. I am able to formulate and write clear, precise and measurable objectives. 

4. I am able to develop test items and other forms of assessment to determine whether 
my students have attained instructional objectives. 

5. I am able to calculate basic descriptive statistics. 

6. lam able to apply basic statistics and measurement principles to interpret and use 
tests and assessment results properly in my classroom. 
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Class participants, in two class sections-001 and 002, were asked to rate their level of knowledge 
or skill in the six assessment content areas, as indicated in Tables 4 and 5, using a 5-point Likert 
scale (pre-assessment). The response scale was as follows: 5-very strong, 4-strong, 3-average, 2- 
weak and 1-very weak. The class participants were also asked to rate their level of knowledge or 
skill in the six assessment content areas at the end of the class (post-assessment). 

Demographically, the class participants were predominantly female, multi-ethnic 
and -racial, with approximately half of the class with less than two years teaching experience, the 
other half with more than two years experience and a maximum of five years. Approximately 
one-third of the students had never taught before. Those who had teaching experience had taught 
in early childhood and elementary grades. Only two teachers had taught at the secondary level. 

As indicated in Table 4, for the pre-assessment phase of the course, the students in 
course section 001, rated themselves in the weak range (i.e., < or = 2) in each of the six 
assessment content areas. The assessment content areas rated the highest, 2.7, for the pre- 
assessment phase were: Item 2, “I am knowledgeable of the role of measurement and assessment 
in the instructional process”, and Item 3, “I am able to formulate and write clear, 
precise and measurable objectives”. As shown in Table 5, for the preassessment phase of the 
course, the students in course section 003, rated themselves in the weak range in each of the six 
assessment content areas. The students assigned the highest rating, 2.9, to assessment content 
area. Item 3, “I am able to formulate and write clear, precise and measurable objectives”. For 
both sections 001 and 003, the assessment content area rated the lowest was Item 6, “I am able to 
apply basic statistics and measurement principles to interpret and use tests and assessment results 
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properly in my classroom”. Sections 001 rated Item 6, 2.0 and section 003, 1 .9. These findings 
are congruent with the studies of Ward (1980) and Plake & Impara (1998) that teachers lack the 
competencies necessary to assess students’ progress in the teaching-learning process. 

For the post-assessment phase of the course, as indicated in Tables 4 and 5, students 
showed much improvement in the ratings of their knowledge and skill in the six assessment 
content areas, with ratings in the above average to strong range. For course section 001, the 
assessment content areas rated the highest, 4.2, were Item 3, “I am able to formulate and write 
clear, precise and measurable objectives.” and Item 4, “I am able to develop test items and other 
forms of assessment to determine whether my students have attained instructional objectives”. 
The assessment content areas rated the next highest, 4.0, were Item 2, “I am knowledgeable of 
the role of measurement and assessment in the instructional process.”, and Item 4, “I am able to 
develop test items and other forms of assessment to determine whether my students have attained 
instructional objectives”. For course section 003, the assessment content areas rated the highest, 
4.1, was Item 3, “I am able to formulate and write clear, precise and measurable objectives”. The 
next highest rating, 4.0, was Item 4, “I am able to develop test items and other forms of 
assessment to determine whether my students have attained instructional objectives”. Students 
in both sections 001 and 003 rated lowest, 3.3, Item 6, “I am able to apply basic statistics and 
measurement principles to interpret and use tests and assessment results properly in my 
classroom”. The students' ratings of the assessment content areas suggests that they were more 
knowledgeable of Item 3, “I am able to formulate and write clear, precise and measurable 
objectives” and Item 4, “I am able to develop test items and other forms of assessment to 




13 



Assessment Portfolios and Teachers 13 



determine whether my students have attained instructional objectives”. It is interesting to note 
that these two assessment content areas were an integral part of the preparation of the student 
assessment portfolio assignment. 

In addition, as part of the evaluation of the course, students were asked what were the 
strengths of the course and areas to be strengthened in the course. Comments were also 
encouraged. For purposes of this discussion, only information relating to the portfolio 
assignment will be discussed. As shown in Table 6, students in sections 001 and 003 made both 
positive and negative qualitative comments. Overall, out of 39 students responding to the course 
evaluation, most of their comments were favorable toward the student assessment portfolio. 
Thirteen students stated that it was helpful; and fourteen indicated that they were more confident 
in constructing goals, objective and test items. The negative qualitative comments made by the 
students were that the portfolio was time consuming, difficult and tedious. These findings are 
consistent with Wolfs (1996) findings on developing teacher effective teaching portfolios. 

In general, these preliminary data suggest that the student assessment portfolio can be a 
valuable and effective instructional tool used for measurement courses in teacher education 
programs. Future research is needed to quantify students' perceptions of student assessment 
portfolios and to determine the conditions under which this type of an assignment can be most 
effective in increasing teachers' assessment literacy. Also, more work is needed to address the 
practical and logistical aspects of guiding and monitoring students' preparation of student 
assessment portfolios. 
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Table 1 

Format of a Student Assessment 

Portfolio for Teachers' Professional Development 



Decorative Cover Page on a Three-Ring Viewbinder 
Table of Contents 
Introduction of Teacher 

Description of Target Population (age/grade level) and Selected Curriculum Unit or Topic 
Preparation of Intended Learning Outcomes (General Goals and Specific Instructional 
Obj ecti ves-Bloom) 

Preparation of Test Items for Each Objective 
Preparation of Test Specifications 

Preparation of Checklists and Rating Scale to monitor the Attainment of Objectives 
Preparation of Test and Performance Assessment from a Subset of Test Items 
Children's Completed Tests and Performance Assessments 

Reflective Summary of the Children's Performance on the Test and Performance Assessment, 
Attainment of the Objectives and Teachers' Self Assessment 
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Table 2 

Standards for Assessment of Multi-Ethnic and Racially Diverse Learners 



Seven Standards for Teacher Competence in the Educational Assessment of Students (AFT. 
NEA, NCME, 1990) 

Standards for Educational and Psychological Tests (APA, 1974) 

Standards for Educational and Psychological Testing ( APA. 1985;AERA, APA, NCME, 1999) 
Code of Fair Testing Practices in Education (APA Joint Committee on Testing Practices, 1988) 
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Table 3 

Artifacts/Forms of Evidence of Standards-Based Professional Development in the Assessment of 
Multi-Ethnic and Racially Diverse Learners 

Components of Construction and Administration of a WrittenTest and Performance Assessment 
Curriculum Unit or Topic 

Learning Outcomes (General Goals and Specific Instructional Objectives based upon Bloom s 
Taxonomy) 

A Variety of Content- Valid Test Items 
Test Blueprint 

Checklist to Monitor Attainment of Objectives 
Checklist or Rubric for Performance Assessment 

Reflective Summary of Children's Performance on the Test and Performance Assessment, 
Attainment of Objectives 

Reflective Summary of Teachers' Self-Assessment of their Construction and Administration of 
the Written Test and Performance Assessment 
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Table 4 

Teachers' (001) Pre- and Post-Mean Ratings of their Knowledge of and Skill in the Assessment 



of Students (N=23) 





Pre 


Post 


Variable 


M 


M 


1 . 1 am knowledgeable of basic concepts 


2.3 


3.9 


and principles in measurement and 
assessment. 






2. I am knowledgeable of the role 
of measurement and assessment in 
the instructional process. 


2.7 


4.0 


3. I am able to formulate and write 
clear, precise and measurable 
objectives. 


2.7 


4.2 


4. I am able to develop test items 
and other forms of assessment to 
determine whether my students 
have attained instructional 
objectives. 


2.3 


4.2 


5. I am able to calculate basic 
descriptive statistics. 


2.3 


3.7 


6. I am able to apply basic statistics 
and measurement principles to 
interpret and use tests and 
assessment results properly in 
my classroom. 


2.0 


3.3 
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Table 5 

Teachers' (003) Pre- and Post-Mean Ratings of Their Knowledge of and Skill in the Assessment 



of Students (N=16) 





Pre 


Post 


Variable 


M 


M 


1. I am knowledgeable of basic concepts 
and principles in measurement and 
assessment. 


2.8 


3.8 


2. I am knowledgeable of the role 
of measurement and assessment in 
the instructional process. 


2.7 


3.8 


3. I am able to formulate and write 
clear, precise and measurable 
objectives. 


2.9 


4.1 


4. I am able to develop test items 
and other forms of assessment to 
determine whether my students 
have attained instructional 
objectives. 


2.6 


4.0 


5. I am able to calculate basic 
descriptive statistics. 


2.4 


3.8 


6. I am able to apply basic statistics 
and measurement principles to 
interpret and use tests and 
assessment results properly in 
my classroom. 


1.9 


3.3 
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Table 6 



Teachers' Perceptions of Preparation of a 


Student Assessment Portfolio 


Type of Qualitative Comments N 


Comment 


Positive 4 


I learned to make a valid test. 


13 


Development of the portfolio was 




helpful. 


14 


I am more confident constructing 




goals, objectives and test items. 


1 


It facilitated student interaction. 


1 


It defined objectives. 


1 


I was able to write instructional 




objectives. 


1 


I can create meaningful assessments. 


1 


It is hands-on-leaming. 


1 


It allows teachers to strengthen skills 




in constructions of examinations. 


4 


I am able to write instructional 




objectives. 
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Table 6 (Continued) 



Teachers' Perceptions of Preparation of a Student Assessment Portfolio 


T 5 T)e of Qualitative Comments N 


Comment 


Positive 1 


I can follow through in the 




classroom. 


1 


We can apply material we are 
learning. 


1 


I foimd out what needs to be done to 
fairly assess students. 


1 


Practice of writing objectives and 
test items was very important in 
helping to prepare me for the 
classroom. 


1 


It helped in refining my 
planning skills as a teacher and also 
in developing test questions to match 
my lessons. 


Negative 3 


Time consuming. 


2 


Difficult and tedious. 




L. 
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